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FOREWORD 



This report is a comprehensive examination of the research evidence re- 
garding computer-based instruction (CBI) on costs, learning achievements, 
individual differences, and student attitudes* Even though CBI promises to 
have a significant impact upon students' learning, inconsistencies and prob- 
lems with the current research evidence prohibit any conclusions about the 
claimed advantages of CBI over other instructional media* Recommendations 
are made about the methodological considerations involved in evaluating a CBI 
system and the usefulness of this instructional technology* A salient recom- 
mendation from this technical report is that researchers should shift foous 
from examining the inherent superiority of this medium to identifying the 
conditions for using computers in the instructional process. The conclusions 
are intended for military personnel and behavioral scientists interested in 
implementing or investigating a CBI system. 




EDGAR M. JOHNSTON 
Technical Director 
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AN EXAMINATION OF THE RESEARCH EVIDENCE FOR COMPUTER^ BASED INSTRUCTION 
IN MILITARY TRAINING 



EXECUTIVE SUMMARY 



Re qui reman t % 

This discussion will help the military training community understand the 
research literature on the usefulness of CBI as a primary instructional medium. 
The issues presented in this paper will also assist behavioral scientists in 
investigating and understanding the strengths and weaknesses of this educa- 
tional medium. 



Procedure : 

Over 150 papers and articles were examined. Interviews with noted CBI 
professionals were also conducted to obtain information about CBI and the re- 
search evidence* This discussion focuses upon four main areas of CBI research- 
costs 5 learning achievements s individual differences , and student attitudes* 
Implications regarding future CBI use and research were also discussed. 

Findings * 

Consistent empirical evidence does not exist to support or deny claimed 
advantages of CBI over other instructional media for (a) reducing training 
time; (b) reducing life-cycle costs; (c) facilitating students 1 mastery of the 
instructional materials; (d) accommodating individual learning differences; and 
(e) motivating students 1 learning. The lack of empirical support for these is- 
sues is not totally explained by problematic courseware* CBI's inherent supe= 
riority as a primary instructional medium has thus not been established. 

Future generations of CBI do promise to have a significantly positive 
impact upon students 1 cognitive process. This medium also promises to help 
slow learners. And it does not seem that CBI is a dehumanizing instructional 
experience. 

Problematic research procedures were also found throughout the CBI litera- 
ture. Most noticeable of these research problems were (a) confoundings due to 
differences in instructional content; (b) making comparisons with inappropriate 
media; (c) confoundings due to "program novelty effects"; and (d) findings that 
were not replicated. 
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Utilization of Findings t 



Widespread implementation of Oil for all educational situations is not 
recommended. Such indiscriminate use of CBI could lead to the abandonment of 
this instructional technology, because the research evidence has indicated that 
at present the promises may outweigh the potential, CBI might be useful as an 
instructional tool to make the established instructional program more effective 
CBI researchers must help determine those situations for which instructors can 
best use this medium. And the methodological recommendations discussed in this 
report should be incorporated in future CBI evaluations. 
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AN EXAMINATION OF THE RESEARCH EVIDENCE FOR COMPUTER-BASED 
INSTRUCTION IN MILITARY TRAINING* 



Introduction 

Computer-Based Instruction (CBI) proponents (e.g., Kearsley, Hunter, & 
Seidel, 1983; Myers, 1984; Shavelson at al ., 1984 and Wilson, 1984) have 
suggested that this technology's delivery capabilities make CBI superior to 
other instructional modes. As Shavelson et al • stated t 

Federal and state policy makers all contend that recent techno- 
logical innovations most notably the microcomputers hold par- 
ticular promise not only for pulling education out of a rising 
tide of mediocrity, but also for reshaping education, (pi) 

CBI denotes using computers in the instructional process, Computer- 
Assisted Instruction (CAI) and Computer-Managed Instruction (CMI) are consid- 
ered to be components of CBI, Si CAI, students receive ins true bion directly 
from the computer system. CMI involves using the computer for such instruc- 
tional management issues as scoring * recording, and interpreting test 
results. Instructions for a CMI system, however, are provided off-line 
either by the teacher or by other instructional media. Other uses of CMI 
will be discussed in succeeding sections, 

CBI is especially being touted as a significant instructional device for 
the United States 1 Armed Services (see Department of Defense [DoD]) . Memoran- 
dum from the Assistant Secretary of Defense, 9 September 1985), Orlansky 
(1985) has noted that CBI is particularly applicable in training military 
personnel for a significant number of the over 10,000 Armed Services 1 skill 
training courses offered in such areas as electronics, maintenance, and com- 
munications. These courses will train approximately 1,4 million students at 
a cost of nearly 4.4 billion dollars in fiscal year 1986, The Army is plan- 
ning to develop CBI programs for training different levels of troops in many 
different types of courses, including advanced courses for officers, Che 
example of the Army's plans to use CBI is the approximately 420 hours of the 
Job Skills Education Program (JSEP), which will be put on computers (Farr, 
1985)* The JSEP package will involve the use of 200-300 terminals located at 
19 different military sites, This program will provide remedial training for 
soldiers with problems mastering the skills needed in their Military Occupa- 
tion Specialty (HQS), The Army 1 s expanded plans to use CBI also involve the 
US Army's Armor School at Fort Knox, The Armor School is in the process of 

•Portions of this report will appear as a chapter in R,T, Hartson and D, Hix 
(Eds) , Advances in Human-Computer Interactions , Ablex Publishing Corpora- 
tion : Norwood, New Jersey* 
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developing over 350 hours of courseware, which rftpresent approximately 20% of 
the Armor School's instructional time (Major P. Carroll 5 DOTD, Personal Com- 
munications 20 May I985), Major Carroll has also claimed that the CBI course- 
ware will be designed for different areas of armor training 5 including advanced 
and basic training courses* 



A major source of support for CBI use in military training is the results 
of research studies which suggest advantages of CB1 over conventional 
instruction. Several reviews of the CBI literature (e.g., Kearsley et al., 
1983; Kemner-Riehardson, Umos, £ West, 198*1; Crlansky & String, 1981 1 and 
Orlansky, 19831 1985) have indicated that this instructional medium has led 
toi (a) reduced training time and (b) reduced instructional costs. These 
reports have also indicated that students seemed to favor taking a course by 
means of CBI rather than receive conventional instruction. The CBI litera- 
ture for civilian training also suggested that CBI training would be the most 
effective instructional mode for helping students to achieve a better mastery 
of the instructional content and for dealing with individual learning differ- 
ences. The robustness of these findings for both civilian and military 
training will be analyzed in this paper. 

A major weakness in the CBI literature is that very few reviews have 
dealt with analyzing the possible methodological problems in this research 
area. This paper will explore the extent to which previous research has 
confounded media effects with extraneous treatment effects. Avner (1978j 
1979) and Clark (1985a) have noted that CBI is a medium rather than an in- 
structional treatment and thus researchers must determine the unique educa- 
tional contribution inherent in CBI as a primary instructional medium. As 
Avner (1979)* Director of Evaluation at the Computer-Based Educational Re- 
search Laboratory at the University of Illinois (CEREL) , has stated, "Com- 
puter-based education (another term for CBI) is, despite its title, more a 
medium for communication of information than a coherent instructional plan or 
approach - " ( p 1 36 ) - 

Without a thorough analysis of the evidence on CBI as an instructional 
medium, one is ill-equipped to make recommendations and decisions about the 
use of CBI as a primary instructional medium for military training. Military 
educators must know if the courseware associated with the CBI system can be 
more or less effectively delivered by other instructional media. This review 
will then examine the state of civilian and military CBI research on costs, 
learning achievements, individual differences, and student attitudes. These 
four main areas of CBI research will be discussed in the next four sections, 
respectively. Methodology concerns in CBI research and implications for 
future CBI research will also be discussed. The civilian CBI research is 
discussed because decisions about using CBI in military training have been 
based upon this research literature. 

Qie must also carefully examine the CBI literature because history has 
shown that educational innovations which were implemented without sufficient 
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research and planning were always abandoned for still newer educational inno- 
vations (Montague & Wulfeck, 198*1), Ragsdale (1982) has suggested that the 
history of CBI may be following the same path as other recent educational 
innovations, e,g a , programmed instruction, from initial enthusiasm and 
excitement for the innovation to its eventual abandonment* With the sizeable 
financial investment associated with large-scale CBI implementation, the 
military can ill afford to implement and then abandon this instructional 
medium because of poor research and planning* 
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Costs 



A major obstacle to widespread CBI Implementation is the extensive 
capital investment associated with acquiring a system. Beed (1 983) cited an 
EDUCOM Bulletin (1983) report which estimated that institutions should expect 
to pay between $1,000 and $6,000 per student for a 1985-1990 CBI system, 
Hofstetter (1983) has noted that the per terminal hardware costs for a PLATO 
(Programmed Logic for Automatic Teaching Operations) system is $2,900. Avner 
(1986) has noted, however, that hardware costs are substantially lower in 
1986 than in 1982 from a $128,000 yearly investment to a $33,760 investment 
for the terminal hardware associated with a standard 32 terminal PLATO sys- 
tem. However, other costs associated with the PLATO system, e.g., costs on 
communication systems, have doubled and tripled in the last four years. He 
has concluded that an institution can expect a yearly investment of $135,000- 
$150,000 for an entire system * These costs reflect the charges from a non- 
profit organization (CEREL) and for the following items* 1) thirty-two termi- 
nals; 2) power for terminals ; 3) terminal site; U) disk storage space* 5) 
system interface; 6) central computer; 7) system software; 8) "central 'build- 
ing; 9) service travel; 10) support personnel; 11) support equipment; 12) 
supplies and spare parts; and 13) communication system to host computer* 
Avner 's figures do not reflect expenditures for courseware materials nor for 
instructors. 

Hardware costs for MicroTICClT (Microcomputer Network of the lime-Shared 
Interactive Computer- Controlled Information Television System) are approxi- 
mately $250,000 for a ten terminal system* This price estimation includes a 
host terminal, color terminals, videodisc players, and a hefty charge for 
lifetime authoring privileges. These costs do not reflect expenditures for 
producing courseware materials nor for classroom personnel* 

Christiansen (1985) observed, however, that the cost value of any product 
should be determined by its life cycle costs rather than by its initial out- 
lay costs* As indicated previously, many CBI reports (e*g*, Hofstetter, 
1983; Kemner-Riohardson et al,, 1984; Orlansky, 1985; Orlansky & String, 
1983; and Seidel, 1980) have suggested that such systems are cost effective 
(same effectiveness at reduced costs or more effective at the same cost than 
alternative media) and would eventually lead to savings in educational expen- 
ditures* Kemner-Riehardson et al* noted, for example, that the PLATO system 
as compared to conventional instruction would lead to a $180,000 a year sav- 
ings for an average military educational program. Such analyses of life 
cycle cost savings attributable to a CBI system traditionally have involved 
estimating operating costs and personnel costs which would accompany imple- 
mentation of new programs. Life cycle cost estimates for military systems 
have also involved calculating the dollars saved by reducing military train-, 
ing time* 
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Training Time 

A moat pervasive finding in the CBI literature, especially the military 
literature, is that use of CBI leads to reduced training time* Orlansky 
noted that the median savings time for 19 military studies when compared to 
conventional instruction was 30 percent with a time savings range of -31 to 
89 percent. These studies have also failed to find any noticeable decrements 
* in students 1 learning associated with spending less time in instruction. 

Dallman, Deleo, Main, and Gillman (1977) reported that such a 3Q% time 
reduction would bring about a yearly savings of $23,000 for an Army course on 
vehicle repair which trains approximately 375 students per year* Orlansky 
(1985) believed that a 30$ time savings could translate to a yearly financial 
savings as large as $10 million in FY 1977 money for training 50,000 Navy 
students. Both Dallman et al's and Orlansky 1 s cost estimates are based upon 
calculating students* pay and allowances for the amounts of time saved in 
training • 

Training time savings could further reduce military expenditures by 
allowing more students to complete a course without the military's having to 
hire additional instructors. A three week savings in a ten-week course would 
allow the military to teach the same course a few more times each year with- 
out hiring additional staff. Dallman et al. f s (1977) and Orlansky 1 s (1985) 
cost savings estimates may then be conservative* 

CBI and conventional instructions, however, are not equivalent media for 
measuring students 1 training time . Avner, Moore, and anith* s (1980) argument 
that CBI should be compared to other self- paced individualized instructional 
media, e.g., programmed texts, is especially relevant for training time data. 
CBI and other forms of individualized instruction allow the students to pro- 
ceed at their own pace, while conventional instruction does not. 

Mixed results have been found when CBI has been compared to programmed 
texts. Time savings of approximately 50 percent for both CBI and programmed 
texts as compared to conventional instruction have in fact been reported when 
all three media have simultaneously been tested (Orlansky, 1985) . Dallman et 
al. (1977) cited a study conducted at the Aberdeen Proving Grounds which 
showed a 10% savings time for CBI when directly compared to programmed texts. 
Other studies (e.g., Dallman et al., and Hemphill, 1986) found, however, that 
students took significantly longer to complete a CBI lesson than to complete 
an equivalent programmed text lesson. Time savings may then not be ascrib- 
able to CBI, but to self-pacing, which characterizes nearly all modern 
instructional innovations. Programmed texts are usually a fraction of CBI's 
cost for initial implementation. According to the Command and Staff Depart- 
ment at Fort Knox, the production costs for a 133 page programmed text are 
between $1.00 and $3*00 depending on whether the book is published by eon- 
tract or by the military. 

Another problem with the CBI literature on time savings iu that CMI has 
been compared to conventional instruction. These time savings effects can 
only be attributable to differences in instructional management techniques 
e.g., time in scoring tests, rather than instructional attributes inherent in 
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CBI. Evaluators should then test the effects of CMI with other (perhaps less 
costly) administrative aids. Again, confounded comparisons have been made 
in measuring time savings associated with CBI. 

A third problem with the time savings data is a lack of replicated find- 
ings. As stated by Longo (1972): 

A paramount notion underlying any research is that of replica- 
tion. In question, of course, is the matter of reliability of 
information across time, samples, and conditions, (p38) 

Without replication and information on reliability, research data have little 
value, Griansky*s (1985) report on the time-savings data does not include 
any description of studies which were replicated for similar instructional 
programs using the same computer system, Also inconsistent findings were 
reported by Orlansky when CBI, regardless of the computer system, and conven- 
tional instruction were compared for similar instructional situations, For 
example, time savings for electronic courses associated with CBI were found 
for Army personnel at the Signal Training School but not for Navy personnel 
at San Diego, 

Operating Costs 

Determining life cycle costs also Includes estimating the hidden costs 
associated with operating a CBI system, Kopstein and Seidel (1969) have 
identified maintenance and courseware production costs as the major variables 
in estimating hidden costs, 

Hofstetter (1983) noted that the maintenance costs of the PLATO system 
would be minimal. He estimated the hourly maintenance cost per PLATO system 
to be 21 cents. This figure is based upon dividing the number of terminals 
into the money included in the contract for maintenance costs, His calcula- 
tions do not consider the system's actual reliability, 

System reliability is important to measure, because repairing malfunc- 
tions will cost additional money in repairing the problem areas and in train- 
ing time losses. Holmes (1982) has indicated that an unreliable system may 
result in severe loss of instructional time as faulty equipment may have to 
be shipped to outside repair centers. A system reliability index— ( failures 
per hour X terminals affected) /(working days by terminals affected) x 100— 
has been developed by Francis, Welling, and Levy (1983). 

Avner (1986) has developed another measure, PI = (# interruptions + mean 
length of interruptions)/total time) or PI = ( 1 + mean length of interrup- 
tions) /(mean time between interruptions), Avner has claimed that these Pi 
indexes are the most sensitive measure of system reliability because they 
reflect the disruptive effect of any interruption upon the instructional 
program* Pi can also be computed for each component of a system (e*g,, 
videodisc failures) and the unreliability of the different components can be 
added to provided an overall system reliability index. 
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Such reliability data for a CBI system must be compared to similar data 
found for alternative programs (Boldovici & Scott, 1981*)* otherwise the data 
are meaningless* Evaluators should obtain information about delays in con- 
ventional instruction - Seidel (1980) s uggested that such information for 
non-CBI programs has rarely been ascertained* 

Seidel (1980) has also suggested that cost comparisons, e.g,, determining 
a system's reliability, between prototype CBI systems and fully operational 
educational systems are not a fair test of CBI*s potential cost value, CBI 
evaluations are usually conducted for a prototype or a newly implemented 
system, such as Francis et al,'s evaluation. Such evaluations may then not 
be an accurate assessment of a fully fielded system's reliability and corre- 
sponding costs. Evaluations are needed in which reliability data are col- 
lected during the initial assessment period and after a few years of 
operation. Information from these evaluations would help CBI professionals 
to more clearly understand the discrepancies found in a system's reliability 
between initial evaluation and actual use, 

A related question to system reliability is how long a system can last* 
Avner's yearly projections for the PLATO system cited at the beginning of the 
costs section were based upon a ten-year life-span. Except for Avner's data, 
little is known about the average life-span for a CBI system. Accurate in- ' 
formation about a system's potential life-span is needed to better understand 
the life-cycle costs associated with CBI, 

Production cost differences between CBI and other instructional media 
have also rarely been systematically analyzed, Several CBI proponents (e.g, 
Avner, 1982; Burnside, 1985; and Dailman et al, 1977) suggested that long- 
term production costs might be cheaper for CBI than for other instructional 
media. Iheir claims have been based on the assumption that updating CBI 
courseware is relatively easy and inexpensive, Dailman et al, have argued, 
for example, that PLATO'S editing features help to reduce sharply the costs 
involved in updating and modifying CBI lessons, According to Avner, updating 
PLATO lessons is a function of the quality of the original work and the 
volatility of the subject matter, Some PLATO lessons need few modifications 
after eight years of continuous use- Another reason for reduced long-term 
production costs is that the CBI system will allow the same instructional 
materials to be repeatedly presented throughout the system's lifetime, while 
some separate programmed texts may have to be created for each student. Very 
few CBI evaluations have examined the differences in actual expenses (and 
problems) with updating the instructional materials for CBI and for other 
instructional media (Orlansky, 1985), 

Initial developmental costs for a CBI program might be quite expensive 
Walker (1984) reported that the developmental costs for a CBI program might 
range between $2,000 and $100,000 per contact hour. Johnson and Flake (1980) 
noted that it would have cost the university of Nebraska library over $30,000 
to present a one-hour program to 4,000 students, By contrast, similar text 
materials cost a few hundred dollars to produce (Walker, 1984), These expen- 
ditures associated with initial courseware development might then negate the 
previously discussed savings associated with CBI for other production costs. 
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An important consideration in determining eost-ef feotiveness is the 
hourly cost associated with using the instructional medium. Tremendous 
variation exists in the hourly cost of CBI service, Avner (1982) has noted 
that PLATO terminal costs can vary depending on terminal use, software use, 
and distance from the central computer from $1-76 per terminal to $10,9*4 per 
terminal, Okey and Majer (1976) have noted that CA1 costs per student con- 
tact hour are $2*07* The $1,78 and $2,07 figures represents nearly continu- 
ous and daily use of the system while the $10. 9^ figure represents sporadic 
CBI use, None of the figures cited has included the expenditures set aside 
for instructors, Also, comparisons of hourly costs for using CBI have not 
been made with those for other media, 

Himwieh (1977) and Stone (1985) have detailed the financial costs 
involved in developing in-house coursewares Himwioh noted that military 
courseware developers must be formally trained in eonaputer procedures and 
educational practices. He also found that it took around 230 hours to com- 
plete one student contact hour of courseware. Stone* s developmental team 
consisted of five Army Captains, civilian educational specialists, and out- 
side consultants* These Amy personnel also had to take a computer course in 
order to design this courseware, Himwich 1 s project was aborted because of 
financial problems and Stone* s team did not meet its goal of completing 100 
hours of instructional time by the following year, 

Several instructional specialists (e.g., Holmes, 1982| and Montague, 
Wulfeck, & Ellis, 1983) believe that in-house courseware can be developed 
rapidly and successfully, Holmes has argued that a template system would 
allow instructors to design CBI courseware in a relatively short space of 
time and with little knowledge of computer programming* Montague et al, have 
argued that computer-aids can help reduce the time and thus costs involved in 
producing in-house courseware, Cost-effectiveness of such templates and 
computer-aids has rarely been studied.* 

Avner, anith, & Tencaar (1984) did complete a longitudinal observation of 
11*3 independent production groups. They found that production efficiency was 
best predicted by : (a) having a strict deadline time; (b) using software 
authoring tools; (c) having subject matter expertise; and (d) having experi- 
ence with media and individualized instruction methods, The use of software 
authoring tools, which can be difficult for the inexperienced programmer , 
provides the most cost-effective method of improving both production effi- 
ciency and quality, Avner et al. have suggested the use of ESiBASIC to meet 
the needs of most courseware programmers working with the Apple II series 
computers and other computers compatible with this series, Other systems, 
e.g.* MICROTICCIT, however* dictate that a special authoring language be 
used , Thus a successful courseware programmer might have to know several 
authoring languages or programs for just one type of system, 

Avner (1979l 1982) also emphasized the importance of the eousreware de- 
signers* having previous experience with the medium and the authoring lan- 
guage, Avner (1979) noted that an inexperienced team of courseware designers 
(less than a year of experience) took between 185 to 610 hours to generate 
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one student contact hour of instruction while highly experienced designers 
(greater than two years of experience) took between 27 to 180 hours to gener 
ate a similiar amount of courseware. He then concluded tha^ : 

Much of the published data (e,g,. Stoned) on production costs 
for CBE, for example, are based on groups with only about a year 
of working experience with the medium. Such data may grossly 
overestimate production costs possible in steady-state (exist- 
ing) operations by experienced design teams and underestimate 
start-up costs (for inexperienced designers), (p137) 

Avner (1982), however, has noted that U30 hours of experience with the 
TUTOR (the PLATO) authoring language is needed for experienced college teach- 
ers to produce their own administrative and pedagogical structures. He also 
noted that $1000 is the average cost for an experienced university staff to 
program an hour f s worth of CBI material, " This $1000 figure does not repre- 
sent the overhead expenditures found for courseware developed by private 
companies and this figure may be reduced when authoring tools and computer- 
aid materials are further refined, CBI production costs must be figured for 
both "steady-state" and "start-up" costs and must be compared with the same 
costs involved in developing courseware for other media, e,g,, device simula- 
tors • 

Personnel Costs 

Personnel costs usually are determined by estimating the increases or 
decreases in the number of educational personnel which would accompany imple- 
mentation of new programs. As previously argued, training time savings could 
allow the military to train more students without hiring more instructors. 
Assuming that the yearly expenditures for civilian and military instructors, 
including benefits, range between $20,000 to $30,000* then the resulting cost 
savings for not hiring instructors could be in the millions of dollars. 
Also, if CBI is indeed more effective than conventional instruction, then, 
ultimately, teachers may need not be the primary instructional medium. 
Teachers would only be necessary to tutor problematic students and monitor 
operations of the computer system, Again, more students could be taught by 
fewer instructors, 

Burnside (1985) has argued that CBI can reduce the educational support 
staff. His argument is based upon the expectation made by computer experts 
(e.g.. Baker, 1978) that CBI, especially CMI, can accomplish the administra- 
tive tasks usually done by the clerical staff, Dallman et al , (1977) also 
projected that support staff needs would require minimal expenditures. They 
estimated spending only $7,200 a year for support personnel 1 s operational 
maintenance tasks. This figure was based upon three civilians--a GS-9, GS-7, 
and GS-5 — spending 20$ of their time on such problems, Dallman et al. f s 
projection may be conservative as maintenance tasks may occupy more of these 
people *s time than anticipated. Also, neither Burnside nor Dallman et al, 
made any mention of the costs involved in hiring support staff to operate and 
update the system, Avner (1982) has noted that the total overhead for sup- 
port personnel for a civilian 32-terminal PLATO system is $989 (at 1982 mar- 
ket prices) per terminal, This figure includes five full-time administration 
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positions, nine full-time maintenance positional seven full-time operator 
positional and fifteen full-time user service (proctor) positions. Burn- 
side*s and Dallman et al.'s position may then not reflect all aupport staff 
costs while Avner*s data may be a gross overestimation for a military CBI 
implementation. Unfortunately, this author could not find any study which 
provided information about actual aupport staff expenditures for implemented 
CBI systems at a military training site. Nor was any study found which com- 
pared differences in support staff needs for CBI with those of other media, 

Several sources (e.g., Bellinger, 1986; Shavelson et al., 1984 ; Walker, 
1984; and Wisher & O'Harra, 1981) have indicated that CBI may not reduce 
personnel expenditures. Wisher and Q'Hara reported that off-line instructor 
task elements of PREST (Performance-Related Enabling Skills Training) CBI 
program increased rather than decreased instructor dependence, They also 
failed to find any significant performance differences between PREST subjects 
and those who were instructed by conventional courseware. Ihey concluded 
that PREST was not as cost-efficient as was the conventional system. 

Bellinger (1986); aavelson et al. (1984); and Walker (1984) have sug- 
gested that personnel costs should also include expenses aasociated with 
training teachers to be "computer-experts." Shavelson et al. observed that 
using a CBI system in a typical ached necessitated a ataff development pro- 
gram which trained teachers to be thoroughly knowledgeable about computers. 
Walker noted that experiences at the Microcomputer Institute for Educators at 
Stanford University demonstrated that 180 hours of study (eight hours a day 
for a month) was needed to train highly motivated teachers to use micro- 
computers properly in the claaaroom, One hundred eighty hours of study is 
not excessive for training in "computer expertise" as the ADAPT course 

authoring courseware for MicroTICCIT) involves nearly three weeks of train- 
ing at eight hours per day. Walker also noted that civilian schools do not 
have the resources to provide the needed additional inservice training for 
ever-changing computer systems. The Army, though, with its computer literacy 
program may be better able to provide its instructors with this additional 
training. Clearly, such hidden costs aa these teacher training programs must 
be included in any analysis of CBI f s life-cycle costs. Of course, possible 
hidden costs in personnel expenditures for conventional instruction must also 
be examined. 

Conclusions 

Any claims about the comparative costs of CBI and other educational media 
are premature. As discussed, very few studies have actually compared the 
cost-effectiveness of a CBI system with the cost-effectiveness for other 
media. Current CBI cost estimates may have also not included all costs for 
this medium and may have overestimated certain costs. 

Claims about the cost savings associated with using a CBI system may also 
be premature, Training time savings may not be attributable to this medium, 
but to self-pacing which characterizes any Individualized instructional 
program. Conclusions about training time reductions and CBI are problematic 
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because information is not readily available about the reliability of these 
data, Also the cost savings in personnel expenditures associated with CBI 
have not been demonstrated. 

The following methodological concerns must then be incorporated into 
future cost comparisons • If training time is the important criterion meas- 
ure, than CBI must be compared with other self-paced instructional systems. 
Information about costs must be gathered for implemented systems . Seidel 
(1980) has noted that reliable and meaningful cost information will only be 
obtained when the innovation is in a stable, operational state. All poten- 
tial costs for the CBI and the alternative medium must also be identified and 
examined, Cost analyses must be done for each CBI implementation. As Avner 
(1978) stated, "Just because 20 prestigious researchers found CBE (CBI) to be 
cost-effective in their application does not mean it will automatically be 
cost-effective in yours," (p25) 
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Learning Achievements 



As previously indicated a major assumption behind implementing CBI is 
this medium* s positive impact upon students* learning processes. Many CBI 
professionals (e.g., Bangert -Drowns, Kulik & Kulik, 1985; Eberts & Brock 
19B4; Fletcher & Rockway, 1986; Kearsley et al,, 1983; Kulik, Kulik, & Cohen, 
1980; Robinson & Kirk, 198l| j and Suppes & Morningstar, 1969) have indicated 
that CBI has or will lead to the development of superior instructional mate- 
rials. Kearsley et al, stated that, "We (CBI professionals) have ample evi^ 
dence that computers can be used to make instruction more effective or 
efficient in a variety of ways." (p 14) 

Critics of CBI research (e.g., Clark, igS5a and 1985b; and Reinking, 
1985) have questioned the evidence of CBl's instructional superiority, 
Reinking noted that comparisons between CBI and alternative media have pro- 
duced inconsistent findings, These critics have also questioned the implied 
assumption that CBI is an inherently superior instructional medium. 

An examination of the CBI training studies is thus needed to help resolve 
this dispute. Existent training studies have usually compared differences 
between CBI students and conventional instruction students with regard to 
test performance and course completion, This latter measure concerns the 
number of students completing the course. Such studies have been referred to 
as outcome studies (Kulik et al,, 1985). Analysis is also needed of CBI's 
relative efficiency for facilitating transfer. Specific CBI features affect- 
ing students* learning will be discussed in this section, 

Outcome Studies 

Fletcher and Rockway (1986) have suggested that CBI is an efficient in^ 
structional medium for military training purposes. They cited evaluations 
conducted by each service demonstrating CBI*s potential effectiveness. Eval- 
uation of a CBI program for an Army Signal Training Course showed that the 
CBI group achieved higher test scores and less attrition than did the control 
group (Longo, 1972)* 

Orlansky (1985) and Orlansky and String (1983) have provided evidence 
challenging CBI's instructional effectiveness for military training. Learn- 
ing achievement was found to be about the same for the CBI and conventional 
instruction students in 32 of the 48 military eases reviewed by Orlansky and 
String, Fifteen of these studies reported slightly superior learning 
achievement for the CBI programs while one study reported slightly poorer 
achievement for the CBI group. 

Francis et al, (1983) provided another example of the inconsistency 
found in CBI results. They conducted two studies of CBX f s effectiveness for 
training at the US Army Missile and Munitions Center and School. PLATO simu- 
lators were found to be an effective and efficient instructional medium for 
the TOW (missile) Field Test Site Training Course, Data analyses failed. 



12 



23 



however, to find any differences between PLATO students* and non-PLATO stu- 
dents 1 test results for the HAWK Continuous Wave Radar Repair Training 
Course * 



Montague (ig8*4) and Qrlansky (1985) claimed that consistent evidence has 
been found in the civilian training literature for CBI's effectiveness. This 
conclusion is based upon a series of meta-analyses (Bangert-Drowns et al», 
1980j Kulik, Bangert, & Williams, 1983; and Kulik et al. f 1980) showing no- 
ticeable effects for CBI upon students* test scores- Bangert-Drowns et al, 
found that exam scores for secondary school students in a CBI program were -4 
standard deviations higher than were the scores reported for the conventional 
instruction students, This finding suggested that CBI was moderately more 
effective than were the conventional instruction programs. 

Inconsistent data for CBI f s effectiveness are also found in the civilian 
training literature, A close examination of Kulik et al,, f s (1980) data has 
indicated that only one-quarter of the 5*f studies cited found large or medium 
effects for CBI upon college students* exam performance- Alderman* Appel, 
and Murphy 1 s (1978) often cited evaluation of PLATO f s and TICCIT 1 s effective- 
ness for community college students also found inconsistent results. The 
TICCIT programs had a significantly positive impact upon students 1 math 
achievement* However, significantly fewer students completed the math course 
for the TICCIT program than completed the conventional instruction program. 
No significant differences were found between PLATO and conventional instruc- 
tion students 1 test scores and completion rates. These inconsistent findings 
may be due to several factors such as courseware differences or problematic 
research practices* 

Methodological Criticisms 



Clark and his associates (Clark, 1 983 j Clark, 1985a and 1985b; Clark & 
Leonard, 1985 I and Clark & Salomon, 1985) have suggested that the cited in- 
consistent findings in the CBI literature are due to problematic research 
practices. They have suggested that CBI has only been shown to be more ef- 
fective than conventional instructional programs when confounded comparisons 
have been made between CBI and other media* They have also indicated that 
such confounded comparisons are prevalent in CBI effectiveness literature, 
Clark and Leonard noted that 75$ of the studies which were randomly selected 
for their review had serious design flaws* And Clark (1985b) has noted that 
the most probable source of confounding in CBI research has been that differ- 
ent instructional treatments were employed in the computer and control condi- 
tions- 
Failure to match instructional content is a common problem in the CBI 
literature. Alderman* s (1974) plans for the evaluations of TICCIT and 
PLATO described methods for comparing students* achievement for dissimilar 
instructional materials* He has claimed that such comparisons can be made 
when substantial overlap exists between objectives for the different courses, 
Some CBI studies* e.g.. Wisher and 0 f Hara (1981) and those cited by Fletcher 
and Rockway (1986) may have indeed compared similar objectives with substan- 
tially different instructional content . The CBI and conventional instruction 



13 




programs m Wisher and O'Hara's study were designed to remedy reading prob- 
lems. The CBI program consisted of eleven reading modules within the Basle 
Skills Earning System devised by the Control Data Corporation while the 
conventional classroom instruction included reading materials created by 
Memphis State University. The similarity of the instructional content for 
these two reading programs was never discussed. Wisher and O'Hara's inabil- 
ity to find significant differences between programs might be due to differ- 
ences m instructional content. Conversely, the effects for CBI reported by 
Alderman et al. (1978) and Fletcher and Rockway might have been attributed to 
presenting different content and not to any features inherent in the CBI 
program. 

Morrison and Wltmer's (1983) comparative evaluation of computer-based and 
print-based job aids for a military maintenance task is one of the few CBI 
studies which has matched instructional content. They made a special effort 
to make the wording and format of the two programs identical. These two 
programs did differ In that the computer-based program used a branching in- 
structional format while the print-based program used a linear model. Given 
these differences, the positive effects found for the oomputer-based program 
might be due to either the medium or the branching sequence. 

Comparing similar instructional materials also requires that identical 
efforts be made in designing the courseware. Clark (1984| 1985a) discussed 
the possible confounding in CBI research due to differences in designing the 
courseware for CBI and conventional instruction. The most powerful extrane- 
ous factor which CBI studies rarely control for is that a greater effort has 
been made to design the CBI presentation than that required for the compara- 
tive media involved (Clark, 1984). Clark does not, however, provide any 
evidence that instructional effort differences do exist or effect CBI re 
search . 

Evidence does exist that some CBI program development has involved con- 
siderable design efforts. As previously stated, developing CBI courseware 
involves considerable time and effort. Also some CBI programs (e.g 
PLATO'S and TICCIT's) were designed by some of the finest educational 'spe- 
cialists. Bunderson (1981) noted that the prototype TICCIT courseware was 
primarily a function of the analytical theorems and artistic talents of the 
courseware development team at Brigham Young University with some help from 
the community college teachers. The Brigham Young staff did not help with 
the development of any of the conventional instruction courseware used in 
this evaluation. One would expect that efforts associated with the classroom 
instruction design did not approach the efforts involved in the TICCIT 
courseware development. 

Several CBI professionals (e.g.. Gray, Pliske & Psetka, 1985| Montague & 
Wulfeok, I98i»| and Walker, 1984) challenged the assumption that considerable 
efforts were spent developing most current CBI courseware. In fact, Cray et 
al. have suggested that most current CBI courseware is underdeveloped. Walker 
has contended that there is a lack of quality CBI courseware. Furthermore, 
Stone's (1985) and Hlmwioh's (1978) developmental teams may have spent the 
majority of their time on programming and not on courseware designing. 
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Clark's (1984 and 1985a) claims about confounding due to instructional ef- 
forts differences may not be valid, but this potential confounding must be 
examined in future CBI studies* 

Clark and Leonard (1985) described another type of confounding associated 
with not matching instructional content* After reviewing *12 randomly select- 
ed civilian CBI programs, they found that CBI subjects received more instruc- 
tion in completing the lessons than did the control subjects. One example 
provided by Clark and Leonard is a study in which CBI groups have received as 
much as eight minutes more of drill-and-practice time than have the conven- 
tional instruction groups* Practice effects rather than the medium may have 
influenced the achievement gains found for the CBI students* The extent to 
which such additional instructional time is a problem depends, of course, on 
the observed amount of instruction to the amount intended by a system 1 s de- 
signer* And the need for instructor prompting undermines CBI ? s promise to 
reduce costs by reducing instructional personnel. Unwarranted instruction, 
such as additional prompting, should then be another variable measured in CBI 
research* 

Using CBI to supplement the instructional program is another issue asso- 
ciated with additional instruction. Several studies (e.g., Dengler, 1983; 
Jelden, 1985; and Mevarech & Rich, 1985) have indicated that CBI is effective 
when used with a conventional instruction program* Most of these studies, 
however, did not include a comparative group receiving either a similar 
amount of conventional instruction or conventional instruction with another 
educational medium* 

Clark (1983; 1985a) and Clark & Salomon (1985) have argued that "novelty 
effects" may also confound CBI research findings* They claimed that the 
positive instructional effects found for CBI in the previously cited meta- 
analyses were due to students 1 initial motivation and enthusiasm to learn 
novel courseware, rather than to any characteristics inherent in the CBI 
system* Clark (1985a) has shown that CBI effectiveness may be very different 
for long- and short-term studies, Clark noted from Kulik et al, f s (1983) 
data that secondary school students* achievement scores were .56 standard 
deviations higher for CBI groups than for conventional instruction groups 
when the studies lasted four weeks or less. Differences between groups' 
performances reduced to .30 for studies lasting five to eight weeks and to 
*20 for studies lasting more than eight weeks* 

Kulik* Kulik, and Bangert-Drowns (1985) argued that "novelty effects" 
were not pervasive in CBI research. They reported that CBI raised students 1 
achievement scores by *26 standard deviations in the typical long-term evalu- 
ation study and by .3*1 standard deviations in the typical short-term study. 
These differences in standard deviations are not statistically significant* 
Since short- and long-term evaluations were not defined by Kulik et alj using 
Kulik et al. f s data to refute the notion of "novelty effects" may then not be 
valid* Clark (1985a) conceded, however, that "novelty effects," were not 
significant detractors from the validity of CBI studies. The issue of nov- 
elty effects will be further explored in the section on student attitudes. 
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Duratiom problMs may exist for military training studies, Orlansky 
(1985) and Orlansky ssLXid String (1983) both noted that half of all CBI mili- 
tary courses lasted e^xie week or lass, Some studies (e.g,, Francis et al.% 
1983) involved limitte^ comparisons for particular course segments, Since 
most military traitiir^^ lasts for a few months, svaluation of less than a 
week f s duration or of" only a few instructional aegmenti are not sufficient 
for making equivalents eeiparisons between CBI and other modes of instruction, 
These limited durations for CBI military studies may work against the possi- 
bility of finding significant differences between CBI and control groups, 

Other criticisms -of* outcome studies include points made about the cost 
studies* For one thi^g # existent outcome studies rarely involved an imple- 
mented systanu Several CBI proponents (e.g.* Montague & Wulfeck, 1984 and 
Gray, 1986) have argu«d that quality CBI programs involves quality implemen- 
tation. And Bundersoaa (1981) noted that the TICCXT courseware used in the 
Alderman et al. ( 1978 3 evaluation was not completely debugged, Outcome stud- 
ies using prototype systems may then provide inaccurate estimates of CBI's 
instructional impact « 

Outcome studies have also rarely been repliea_ted and have rarely involved 
comparisons Kith ©their* individualized instruotion.&l media* questions linen 
exist about the stability f these studies 1 results. And any affects round 
for CBI may then not toe dy to the madia but rather to attributes inhar-ent in 
individualia^d instruction. 

Transfer Studies 

Evaluating CBl ! s E-nstruotional value also involves determining whether a 
system is being used correctly. Avner (1978) has observed that the cor-reot 
usage of the medium is more important than the ftct that it works - Correct 
usage of any edueatiermal system involves helping students to make the neces- 
sary transfer* of information to the appropriate environmental situations , 

ControvafTsy exists^ In the CBI literature regarding this medium's effec- 
tiveness for facilitating transfer . Many CBI proponent's £a,g,, Gleason* 
1981; Kern, T985; McLaughlin, 1985; Montague, 198*1 j Reigeluth 1 Garfield , 
1984 1 and Salomon* 19TS) toe suggested that CBI would help develop the 
learning processes needed for successful transfer • For one thing, McLaughlin 
has indicated that th^ computer is ideally suited for developing both proce- 
dural and declarative level knowledge . Procedural level knowledge is a basic 
understanding of the s ^eps involved in completing a task while declarative 
level knovrleage involves an understanding of the principles and ooncapta 
associated vtLth that t^ask (Clark & Vogal, 1985)* According to MoLaughlin, 
CBI course wr-e should -fchen augment students' abilities to make basic proce- 
dural level transfer— e .g,, following a technical manual to troubleshoot an 
armor system*, CBI WQU3.d also help students with cognitive transfer— e.g . f 
understanding the principles behind the repair, 

CBI proponents also believe that this medium TOuld help students to de- 
velop the mentery skills needed for long-term transfer. One reason for this 
belief is thct drill^arad-praetiee programs would enable students to rehearse 
the information continuously. Continuous rehearsal is a primary way of stor- 
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ing information into memory (Landhauer, 1982)- Also, many computer profes- 
sionals and behavioral scientists (e.g*, Bunderson, 1 981 - Ko-^sslyn, 1981; 
frferrill e*t aim, 1980 1 Olson & Bruner, 197 1 *; and Salomon & G^-ardner, 1986) 
have suggested that active learning and dual-modality proems— sing ire associ- 
ated with computer instruction* Active learning and dual*-tfo clality processing 
can help facilitate deep cognitive processing of information , And informa- 
tion processed into deep cognitive levels is resistant to fo -rgetting (Craik & 
Loekhart , 1972)* These elements of drill-and-practice , act! ^ve learning, and 
dual-modality processing often found in CBI programs would t— lien seem to fa- 
eilitate a mora permanent transfer of Information than would be offered by 
other instructional media* 

Clark and his associates (Clark & Salomon, 1985 and Clsr* Is fitVoogel, 1985) 
challenged the assumption that CBI would augment students* t ^ansfer abili- 
ties, Clark and Voogel have suggested that current CBI cour seware is geared 
toward helping students to make procedural level transfer bu % not cognitive- 
level transfer. Bialo and Erickson (1984), after evaluating the courseware 
for 163 microcomputer programs, found that 70$ of the course- "ware dealt solely 
with procedural level objectives. Clark and associates cor^^espondingly have 
indicated that CBI has minimal influence upon students' long- —term transfer 
and memory process, Cognitive effects cited to be assgoiate»<l with CBI, e.g., 
deeper processing of information, are not necessarily unique to this medium 
(Clark & Salomon)* 

Limited research evidence has been collected about CBl f s potential effec- 
tiveness For facilitating either cognitive processes or trsn^^fer, Gleason 
(1981); Mo Donald and Crawford (1 983); and Tatsuoka and Misse„lt (1978) have 
suggested that retention following CBI training is substantival * As stated by 
Gleason t "Retention following CAI is at least as good if no™fc superior to 
retention following conventional instruction." (p16) NevertI3ieiesii Gleason 
does not provide any data to verify this claim. Furthermore , McDonald and 
Crawford failed to compare the retention obtained by CBI 1 ^ amabjeota with a 
control group's retention performance, and Tatsuoka and Wi^eelt'a findings 
seem to be an artifact of the lesson f s being inadvertently r*«peated because 
of hardwar'e problems. 

Several other reports (e.g., Swezey, Criswell, Huggins, XHays, & Allen, in 
press; Swezey, Huggins, & Perez, in press; and Pagliaro, 198^3) failed to find 
any significant effects for CBI upon either subjects 1 transfer r or retention 
performanQe* Pagliaro found that pharmacology students' ret^sntion perform^ 
ance was the same for both CBI and conventional instruction "retraining- Swezey, 
Criswell et al. found that practice on a 3*D module eomponsn^^ was more in- 
strumental than was the instructional media (CAI, CRT or vid^^otape) in fa- 
cilitating military students 1 transfer on a simple engine regpair procedural 
task. The same results were replicated by Swezey, Huggins, usand Perez for a 
eomplex procedural task. CBI f s potential for facilitating r^stention of 
procedural tasks must be further examined because military OBI programs are 
frequently used for such tasks* 

Personal experience has indicated that incidental learni^^g lean impor- 
tant part of conventional instruction. Students, for instafi^se, in an educa- 
tional psychology class may incidentally learn from the teaefaier's mannerisms 
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ana class discussions teaching techniques which may last a lifetime. Inei- 

^h a ^ni?H rn ff ! S a "° ther as P eet " of transfer iioh has rarely been examined, 
Sucsh incidental learning may indexed not b e po S ilble with CBI because of pro- 
gramming limitations. In this cl-se the spontaneity and richness found in 
classroom environments may not be completely duplicated . Limitations in this 
medium may then be limiting the courseware ana not the courseware limiting"'" 
tha. medium as previously indioatew<3 , However, son important incidental 
learning may be occurring in CBI wwhieh evalustora are not measuring . For 
instance, a CBI military training course on maintenance may also help the 
?rSSf? e m, be ;ifn b % tfcer readeM -- Even though insuring Inoidental learning 
nntJS hi r«' Tf tl f/ s 0 nwd « d about CBJ'slmpaet upon such learning. As 
netted by Reeves and Lent (1 98 2), -^The sponsor's mi consumers of OBI should be 
inf-ormed as much as possible aboutJb the ultimate effects of this medium when 
the^r make decisions about eontinuaanee, expansion, or selection." (pl3) 

Human Factors and Effectiveness 



Sawyer C 1985) noted the importance for attaining human factors issu 
CBI evaluations. According to Sawsryer, evaluating human factors issues x.xi- 
volwes examining the effects of a system's h a r dl r e a „d documentation upon 
atuaents f performance* 



s in 



Kearsley et al. (1983) have claimed that a CBI system's response time and 
dowrmtime are important factors in the success of a CBI project , These eSimen- 
8 ;f' Bay * ffeet user's mastery of - the instruflfclonal information. Tatswoka 
and Masselt ( 1978) noted that mechanical failures with the PLATO system Had 
negatively effected US Air Force students' leaning efficiency. Educational 
experts not in the CBI field have -^stressed the Uportanee of measuring the 
rioter P f eduoational Productivity Wda medium's downtime! Sump 
(19SS) suggested, for example, thamt a downtime of 10$ in a conventional in- 
struction program could «qual an Instructional efficiency loss of 101. Com 
pardons between CBI am* other instructional tf&dli should examine potential 
instaructional loss due to "downtime" problems, 

"The civilian computer literutm — e (e.g., QttjMis, 1985: Card, English & 
flurry 1978; Phillips et al., 1984 ; and Reigelutj, | Garfield, 1984) "has dii- 
cuss*3d the impact that certain eQnp-mter hardware features have upon users' 
performance. Card et al. found sissnifioant dlffirenees in subjects' abili- 
ties to use four common computer r— esponding m6otanisms.-a mouse, a joystick 
step keys, and text keys— with the mouse showing in the best results (see 
Card et al . for a description of thtmese devices) . Ohapanis noted that people 
were normally able to read material's presented onahard-eopy 30$ more effl- 
cienfcly than when they were present _ed on a CRT* Phillips et al. reported 
that problems with screen visibility on CRT's landed the educational per-- 
formatnce of secondary school students. They also found that awkwardly de- 

llfST keyboards and terminals poor, ly placed in classrooms led to educational 
pro dj — .€qs ■ 

documentation problems may also adversely WAot CBI's education poten- 
tial. One cannot fully understand wior use any instructional procedures with- 
out adequate source mntcria Is. BiaD.o and ErioKs on (198H) found the 
documentation to be barely adequate for most of tie 163 microcomputer systems 
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reviewed and information about instructional materials was absent or* inade- 
quate for 62$ of these programs* Chapanis (1965* 1985} also found dooumanti- 
tic^>n materials for eomputers to be inadequate. He has shown that reading a 
system* s documentation requires the reader to be familiar not only with nor- 
maE, computer jargon but also the particular system 1 s jargon. 

Conclusions 

" .' I M 111 — li_ -M_ ML . J — JJI 

Inconsistent findings have been obtained about CBI f s potential impact 
upc^n students* learning, and positive instructional effects found for CBI 
mi^ht be traceable to factors other than the medium* Most pronounced among 
ttese non-medium effects were practice differences and instructional content 
dierferenees between CBI treatment and that of the comparative medium » As- 
auEnptions about eomputers making instruction more effective have thus not 
baesji validated, Human factors and courseware problems, however, may limit 
th» oomputer f s potential effectiveness, especially for facilitating cognitive 
akSLXis, A basic unresolved question in CBI research is whether possible 
prcsblems with CBI are due to such courseware and human factors limitations or 
dues to the limitations of the medium* 

CBI researchers need to obtain more conclusive data on the long-term 
effects of CBI, A primary function of any educational program is to insure 
tjiat long-term changes occur within the students* The military is especially 
iiiteerested in troops retaining skills and knowledge for an indefinite period 
of time. 
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CBI and lndivi_dua] Differences 



Handling individual learning differences has long been a difficult prob- 
lem for educators. Many CBI profesaio— nals (e.g., Avner, et al,, i960- Car- 
rier, 1979; Kearsley et al„ 1983; Mer-riil et al., I98O; and Steinberg, 1984) 
believe that this medium is ideally aa ited to handle individual learning dif- 
ferences. As stated by Carrier, "To m mtiy educators, the computer represents 
the ultimate individual difference mao™h±ne (p25) 

CBI* 3 superiority over other media would then be moat pronounced for 
meeting the educational needs of all 3— fcudents, Arguments supporting this 
position have come from CBI reports on learner control mechanisms and cogni- 
tive differences. Reports by Carroll <1984); Hess & Miura (1985); and Wood & 
Fitz (1985) have suggested, however, tKiat some students may have problems 
learning from a CBI program. 

Learner Control Mechanisms 

A basic belief held by many educators is that students should be encour- 
aged to make educational choices for tfc- iemsalvea (Carrier, Davidson, Higgson, 
& Williams, 1984). Such choices would enable students to more actively proc- 
ess the instructional material, As previously indicated, such active learn- 
ing would lead to superior learning. Learner control is also expected to 
help educators deal with individual learning differences. Gay (1985) has 
notod that: 

It is commonly assumed that individual differences in abilities 
and aptitudes will be accommodated If learners have more control 
over the pace, amount of practice, or style of instructions they 
receive, (pi) 

CBI designers have developed lea me- - control mechanisms which provide 
students with some degree of freedom in selecting the instructional materials 
to study. TICCIT students, for example , can select, through a special key- 
board, the display beat suited for thei— educational needs (Merrill et al 
1980). Merrill et al. have also olaime-d that this learner control mechanism 
differentiates TICCIT from other instructional media, including PLATO and 
some other CBI systems. 

Some research evidence (e.g., Collier, Poynor, O'Neill, & Judd, 1973. 
Fishman, 1985; and Hansen, 1972) exists supporting the use of learner control 
mechanisms over program control (computer controlled sequences) . Hansen 
found that learner control subjects exhibited greater decreases in state 
anxiety than did control subjects. Collier et al . also found reduced anxiety 
levels to be associated with using leare-ner control mechanisms . Increased 
learning usually occurs when anxiety tofe-*ard the instructional material is 
reduced . Fishman observed that learner control mechanisms allowed students 
to have the additional practice needed t=o most effectively master the mate- 
rial. She found that learner control sfc=udents outperformed students who 
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received instruction from lecture and video presentations! Perhap^s then* 
learner control meQii&&nisms can make CBI a unique and superior instructional 
medium. 

Several reports (Ile.g., Ford, Slough, & Hurloek, 1972; Gay, 19S5; Judd, 
Daubek, & O'Neii, 19TSI5s Snow, 1980; Steinberg, 1977 1 and Tennyson & Buttrey, 
1980), however, have challenged the previously cited research aboi^t the in- 
structional ef feotlyesness of learner control features* Steinberg in a com- 
prehensive review of this literature concluded that students who were given 
complete control over-" course flow took longer to complete the ooiir^se than did 
control students* Aliso, experimental and control students scored equally 
well on achievement treats - Ford et al* did not find any improvetfe^zits in Navy 
students 1 learning inj^ the Basic Electricity and Electronics SchGo!!_ associated 
with using learner d&ontroi mechanisms* Tennyson & Buttrey noted trmat when 
students controlled k=he instructional process, they often termihp^ed the 
process too early arid S did not effectively learn the materials, Juffltd et al, 
furthermore, conclude ^d that a complex interaction existed between subjects 1 
ability to use learw^r control mechanisms and the subjects 1 characteristics* 
Students with high AO ^hievement- Independence scores- -as measured py^ the Cali^ 
fornia Psychological Inventory™were the students most able to u^e^ the learn- 
er control features* 

Gay (I985) found that learner control mechanisms did not acQdnmmodate 
individual backgrounds differences* Subjects with high prior conceptual un- 
derstanding of the ma mterial made significantly better use of theses control 
options than did 3Ubj*leots with poor prior understanding. ReinkirJ^ (1983) and 
Carrier et al* (I984) * also found that control options did not aooo**mmodate 
individual learning d fiif ferenees • Reinking found that the use of L earner 
control options did n*not improve poor readers 1 level of reading* Lil students 
may then not be able to use and benefit from learner control optic*- 21s- 



Cognitive Differenggs m 

If CBI T s instruoti^ional impact was not found for the learner co mtrol is- 
sue* it might then b^ - discovered by examining the cognitive differ— mncmm lit- 
erature. Many repQrU^s (Allen & Merrill, 1985; Bangert-Drowns et a__Jl 1985 1 
Farr, 1985; Gray et a/jiX** 1985; Neimiec & Walberg, 1 985 ; and Zmkm, 198^) have 
suggested that the maixin reason for selecting a CBI system is to be ^bter deal 
with differences in Oa-ognitive abilities and styles* 

Most studies e^nos-ernihg disadvantaged learners have used highlz3r struc- 
tured drill-and-pract;=ice programs (see such projects as the Clevel-^and Public 
Schools, 1981 and Mevs^areoh & Rich, 1985)* Such courseware seemed t^o provide 
poorer learners with t zhm needed additional structure of and practi^^ee with the 
laaterials. As Bangert*t- Drowns et al* discovered , CBI had the moat *«ffect upon 
increasing low abilit^.y students 1 test scores than for any other ua-^ge* The 
meta-analysis compares son between CBI and conventional instruction revealed an 
effect size of J6 in . favor of CBI for teaching low ability sub jeQ™ ts * Zerake 
(198^) also observed * that low achievers seemed to gain a great dealZL from CBI. 
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Problems do exist with selecting a CBI program based on this literature. 

noted'tLf rif (1985) * MUa ' Ba3S ' and Sharpe (1986); and Zemke (19 
noted that CBI programs were the most effective when used to supplement the 
classroom instruction. As stated in the previous section, many of the sup 
plemental studies did not include a comparative control group. itapro^enS 
in disadvantaged students' learning could have then been produced by using 
any instructional medium to supplement the conventional instruction program, 

Drill-and-practiee programs might not be useful for all subjects. Such 

^nr e , m f ht 'I t0 ,° Str " CtUred and limlted for »° 3t atudentf, eflpeeany 
gifted students. B. Clark (1983) has concluded that gifted students need 
tasks which allow them the freedom to proceed beyond f rudimentary rehing 
of the materials. Since drill-and-praetioe programs dominate the market 
most current CBI programs may have limited value for gifted students. 

in^ D H« ta /r° m * §tUdy oonducted b y Baraam & Simutis (1984) provided some 
insight into courseware which might be geared toward certain kinds of rifted 
students. This study explored the relationships between individual di er 

S ?!!J abilities and soldiers' performance on a CBI terrain visu- 
alization task consisting of three-dimensional graphics. This study also 

d 'f h feren0 f 3 ^ abilities to benefit from courseware ieh 

utilized either active (subjects select the graphic for viewing) v a laSve 

TneffouL1hafth th ^ S r Phi ? T ^ objects' 'viewing) training ^eSrel. 
They found that the high spatial ability subjects were better able to com- 
plete the lesson successfully and benefit from the active training procedures 
than were soldiers with normal and low spatial abilities. 

The high-spatial ability students seemed to have learner characteristics 
similar to "field-independent" students (Witkln, Goodenough, 4 Rarp 1Q ) 
who are usually the high achievers in math and science corses. [lee Sin 
Moore, Goodenough, & Cox, 1977 for a further description of the relatio hJp 
between field-independence and math/science aptitudes). Field-independe t 
students are also quite analytical in viewing three-dimensional objSstd 

1 , f ' fr ° m ^structuring the instructional environment? Corre- 
be t^frX', S ! Udents with mathematical and scientific aptitudes would 

™ „1 " ° SU ° cessfull y h ™ d ^ CBI lessons, e.g., simulation pro- 

"J*? ™ me ™«? oomplex graphics. These last few paragraphs suggest 
that: (a) the basic drill_and_praetiee CBI programs favor the poorfr 
learners while (b) the advanced CBI simulation courseware is geared toward 
the more "gifted students," especially in mathematics and science. 

« yr^ 11 ^ in student3 * abilities to use different types of CBI programs 
V?r/f atC . t0 COenitlvC st ^ di^rences. Witkin et al. (1977 
claimed that individual differences along the field independence/dependenoe 
continuum reflected basic differences in people's characteristic styles 

2S 2 fSent I TiI^T^'t^ eontlnUura does not differentiate students 
« 1 ^ Q " levels. It does differentiate students with different 

modes of handling instructional materials. Field-dependent people rely on 
KTSS fTT' teacher " s ' to structure and help understand the C L 

indenendV? and abstract stimuli, e.g., computer graphics, while held. 

1 d" 60 ! j need suoh eternal aids. And, as previously indi- 

cated, field-independent people are better able to more effectively use the 
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information inherent in such ^situations- Witkin ©t al. have also observed 
that instructional situations with explicit instructions, concrete presenta- 
tion of stimuli, and explicit information about performance outcomes favor 
field^dependent students* M^^ld-independent students are better able to 
perform on more abstract and Mess structured tasks* One would then expect 
that drill-and-praotiee CBI S^rstems with structured instructional sequencing 
and continuous perfonanee fe^edbaqk would be the most appropriate for field* 
dependent people* CBI progr^^ns with complex graphics and learner control 
features* *as described in the pre virus paragraph8™would be more geared to- 
ward field-independent people 

Limited and contradictory evidence exists about the actual relationship 
between current CBI programs c~tnd cognitive styles. Carrier et al. (198^3 and 
Park and Roberts ( 1 985) found data which^ indicated differences—as expected" 
in CBI performance aaaooiated with cognitive style differences* park and 
Roberts found that fisld-dep#i^dent people did need more guided instruction t© 
complete a set of CBI lesaonp than did field-independent people * Carrier et 
al, however, did not find diffiTerenoes between field-independent and field- 
dependent subjects in their Me of control options* Burger ( 1985 3 did not 
find any significant ralatio^^ships between students* cognitive styles and CBI 
preference and performance. 

An instructional system* ^ adaptability to a students* cognitive style is 
a major issue addressed by WA^skin et al. Indeed, they emphasized the need 
for the educational iidium to match the learners* cognitive style* Several 
CBI advocates (e.g., Kmrml^f*^ et al., 1983; J* Olsem* personal communica- 
tion, October* 1985; and Ten.tf?^son, Christiansen* and Park* 1984) have sug- 
gested that the new generafciejnaa of CBI systems with intelligent programs 
(smart technology) can make tide instructional system adaptable to students" 
learning styles. Olseffl has eliaimed that these ICBls (Intelligent Computer- 
based Instruction) have the c^mpaoity to modify the instructional mode to best 
meet the students* learning ff^^ofiles. These learning profiles are determined 
by cognitive tests embedded iro the first phase of the ICBI*s instructional 
program. 

&iart technology Qin ala$ be used for accommodating students with differ- 
ent cognitive abilities (A11#JD. & Merrill, 1985; and Gray et al., 1985). Smart 
technology programs involve irtateractions between students and the computer so 
that the type of feedback and instructional materials provided fit the stu- 
dents* level of knowledge (Qri^my et al.)* They have also claimed that the 
promise of this technology t# that each student's unique cognitive needs will 
be met* Unfortunately, the§# smart technology programs have neither been 
fully implemented nor tested* 

Adaptive control programs— -*a form of smart technology— have been shorn to 
be effective for facilitating students* learning (Avner et al 1980; Ross, 
1984; and Tennyson 4 Buttrey, 1980). An adaptive control is when the amount 
of instruction is managed by t=he CAI program based upon students 1 achieve^ 
ment-level, aptitude, or oognL— tive-style (Carrier et al, 1984) Ross has 
shown that such a program can accommodate individual learning differences - 
Furthermore, Avner ttal# ftave« claimed that adaptive control programs repre- 
sent the real basis for CBl*^ superiority* 
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CMI is another adaptive control mechanism which is seemingly suitable for 
accommodating individual differences (Baker, 1978; Federico 1983- and 
Kearsley et al„ 1983). CMI can be used to advise students and teachers 
about the most appropriate instructional sequence for each student, Federico 
showed that for Navy trainees CMI programs reduced the learning variability 
usually associated with cognitive differences. 

The supportive research for adaptive control mechanisms may be problem- 
atic. Research on smart technology applications have been experiments of 
short duration which have rarely compared CBI to other media. These studies 
have also mainly investigated CBI programs in laboratory- type settings. Ques- 
tions thus remain about the relative effectiveness of smart technology pro- 
grams for educational settings. And, Ross (198*0 has demonstrated only 
limited advantages for CMI over other media for dealing with learning differ- 
ences. CMI was only found to be advantageous for complicated and lengthy 
lessons. 

Problems may also exist with using adaptive CBI programs. CMI programs 
are used in conjunction with either conventional instruction or CAI programs. 
CMI is thus useful as a supplemental but not as a primary mode of instruc- 
tion. Also, Merrill et al . (1980) felt that adaptive CAI programs would 
ultimately be "maladaptive" for meeting students' educational needs because 
students* subsequent learning in non-CAI environments would become more dif- 
ficult as the real world does not adapt to the student. 



Individual Differences and CBI Limitations 

As discussed previously, CBI may not be beneficial to all students. Sev- 
eral investigators (e.g., Cambre & Cook, 1985| Jay, 1981,- Loyd 4 Gressard 
1984a & 1984b,- Maurer, 1983,' and Wood and Pita, 1985) have found that com- 
puter anxiety can inhibit a person's ability to successfully master computer 
skills | consequently, such anxiety may inhibit a student's mastery of infor- 
mation from a CBI system. As discussed by Cambre & Cook, the following four 
behaviors are associated with computer anxiety (p.52)i 

(1) Avoidance of the computer and general area where the computer is 
located . 

(2) Excessive caution with the computer. 

(3) Negative remarks about the computer. 

(4) Attempts to cut the computer session short. 

The existence of and problems associated with computer anxiety have not 
been systematically investigated. Loyd and Gressard (190J|b) have attempted 
to rectify this problem in the CBI literature by devising a self-report ques- 
tionnaire which measures, among other things, these cited dimensions of com- 
puter anxiety. They have claimed that this questionnaire provides an effec- 
tive, reliable, and convenient measure of students' computer anxiety. Wood 
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and Pitz (1985)* however, have suggested that any self-report questionnaire 
on computer anxiety (e*g*, Loyd and Gressard' s,) without scales for lying and 
social desirability may not be a valid measure, because subjects 1 stated 
expectations about using a computer are not always congruent with their ac- 
tual computer behaviors* Personal experiences have also demonstrated that 
people may be reluctant to state their fears of using a computer* 

A reliable and valid questionnaire on computer anxiety would be benefi- 
cial to both educators and researchers* Such a questionnaire would provide 
additional insights into explaining possible variability in students* abili- 
ties to use CBI materials* This scale would also then help educators to 
pinpoint those students who need additional support and guidance for using 
this medium. Schubert (1985), for example, found that computer anxious fe- 
males had to go through a formal desensitization process before they were 
able to deal with the CBI materials* Consequently, the educational, temporal 
and financial costs associated with helping such computer anxious students 
should also be included in any evaluation of a CBI system* 

Differences in computer experience may also differentially affect stu- 
dents* use of CBI materials* Carroll and his associates (e*g*, Carroll, 
1984| Carroll, & fcfesur, 1984* and Carroll & Thomas, 1982) have noted that 
infrequent computer users often have difficulty in learning to use a computer 
system* They also suggested that embedded instructions for using a computer 
must relate to students 1 prior computer use and knowledge. CBI professionals 
must then consider both novice and experienced computer populations when 
designing and evaluating a CBI system. If the system cannot accommodate all 
levels of users, then a segment of students, especially the infrequent users, 
will not be able to use and benefit from the CBI system. There also may be 
extreme variability in subjects 1 training time on a CBI system, with infre- 
quent users proceeding much slower then regular and frequent users* Such 
variability may then negate OBI's previously cited potential for reducing 
instructional time* This variability may decrease in military training as 
the number of military students taking computer literacy and CBI courses 
increases. 

Computer experiences may also relate to computer anxiety (Loyd & 
Gressard, 1984a* and King, 1975). Even though the data may be problematic, 
Loyd and Gressard did find a significant relationship between infrequent 
computer use and computer anxiety. If this is so, then computer-experienced 
users would likely be excited and motivated to use a novel CBI program while 
non-experienced users might be apprehensive about using such a program. This 
novelty issue will be further discussed in the section on student attitudes* 

Hess and his associates (e.g., Hess & Miura, 1985,* Hess & Tenezakis, 
19731 Miura & Hess, 1983, and Schubert, 1985) have indicated that there are 
sex differences associated with students 1 use of computers. Miura & Hess 
have found that middle-school boys more than girls i a) owned and used home 
computers and b) enrolled in computer camps and summer classes* They also 
asked these students to rate a number of randomly selected courseware titles 
as being either male of female oriented, These students rated a significant- 
ly larger number of courseware titles to be primarily suited for male 
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audiences and not for female audiences, other studies e a i 



Conclusions 



Individual differences may conversely limit rRTf e n«^,i 
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CBI and Student Attitudes 



The moat seemingly pervasive finding in the CBI literature is that stu- 
dents favor taking a course by this medium, Many reports (e.g*, Clement, 
1 98 1 1 Merrill et al., 1980| and Myers, 1984) have urged implementation of CBI 
based upon student attitudes. Myers noted that PLATO has been enthusiastio*. 
ally accepted by students and teachers. Also, advertisements on CBI products 
have claimed that "learner motivation is high because the system is easy and 
fun to use," ( Teaching/Learning Technologies, 1984) 

As the research on students 1 CBI attitudinal data has rarely been exam- 
ined, the robustness of the data about students 1 attitudes toward CBI as 
compared to their attitudes toward other media must be questioned. This 
analysis must also include examining the pyschometric attributes of the dif*. 
ferent attitudinal measures used in these studies, In this section, factors 
influencing students 1 CBI views will also be discussed. 

Attitudinal Studies 

Comparisons of students* attitudes toward CBI and other media have been 
included in some of the previously cited military studies (e .g . , Dallman et 
al., 1977; Ford et al,, 1972j Longo, 1972| and Tatsuoka & Misseit, 1978)* 
These studies indicated that students favored taking a course by CBI rather 
than by other media. Longo, for example, found in two separate studies mor^ 
positive ratings being exhibited by the CBI group thin by the conventional 
instruction group, 

Ballman et al, (1977) also reported that students tended to favor taking 
a CBI course rather than a conventional instruction course , They noted, 
however, that the conventional instruction course was taught by newly trained 
teachers* They also found that a conventional instruction course taught by & 
popular teacher was more favorably received than was CBI, In a second stud^r, 
they failed to find any significant differences between students* attitudes 
toward CBI and toward programmed instructions, 

Mixed results about students 1 preference for CBI over other instructional 
media were the pattern for the civilian literature, Avner and associates 
(Avner, 1981 1 and Jones, Kane, Sherwood, & Avner, 1983) in a series of course 
evaluations found that University of Illinois 1 students favored PLATO over 
other instructional media, For two semesters. Chemistry 100 students were 
asked to rate several different media with regards to helpfulness in learning 
the class materials. These media were PLATO, textbook, lecture, labs, and 
qui^ sections. The main professor for this course had received several out^ 
standing teaching honors, For both semesters, the PLATO system received tftm 
highest ratings of 4*5 on a five point scale. Differences between ratings 
for CBI and conventional instruction were substantial—over a point differ^ 
ence on a five point scale. Differences in students 1 ratings of CBI and Qui^ 
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iT^h^n ' l) "? f ° r Fal1 ' 1977 and S P rln S' 1978 semesters, respective- 
ly. These ooursa evaluations and Dailman et al's study are some of the fJw 

a S ttiL1 e s n t o W wa e d S^*' CD ' "" compared with theL 

attitudes toward other non-eon von tional modes of instruction. 

suDDorff^'fh 0 ™ 8 " S' (1985) and Kulik et a1 ' <1980) demonstrated limited 

n f n S Z P r:?- th3t StUd6ntS Pr<:fer - d CBI « B^h meta-analyses 
could only find a few studies— eleven for the Bangert et al, analysis and 
five for the Kulik et al. analysis-which compare! attitude- Swarf Csf and 

oTSSmutt™ 1 ln f rU ° Uon - ******* et 1- f-nd an average effect of 
a e c f P nooondary students' attitudes, and Kulik et alT discovered 
an effect of .18 favoring CBI for college instruction. 

Problems with concluding that students prefer to take a CBI rather than a 
TS ^ruction course are vividly demonstrated by Alderman et " ' 

tive feelinL ££"5 %V ^ peroenta 8 e of PLATO students indicated posi- 

tJ S rard thlB ^raaware as manifested by their desire to take 

E£EZ*S°* m U f nS thl * SyStM * Th8y alSO disagreed that uE£ PUTO was 
dehumanizing or boring. Attitudinal comparisons between PLATO and conven- 
tional instruction students, however, failed to reveal any noticeable differ 
found "f^f/^ents. feelings toward their classes. AlSIrman It A/Ull 
found that students reactions to TICC1T programs were usually less positive 
than those for the conventional instruction program. Interestingly enoJS 

tive i^ haS ^ f dely CitBd SS Riding ^idence that CB^ has a pef ±i 
tive impact upon students' attitudes, p 3 

Magidson (1978) is another study widely cited as demonstrating that stu 
dents prefer to take a CBI course. Magidsen's results about PLATO users^ 
views parallel those found by Alderman et al (1978). Over ninety percent of 

^SrS?^? 7 " 1 "^f PUT ° W ° Uld "««»d that theS IrienSs 
take a P^ATO-based course. He also reported that less than ten percent of 
these students found PLATO to be boring and dehumanizing. "titudinal com- 
parisons between PLATO and conventional instruction stulents were not mS" 

Conclusions cannot then be made about students' preference for CBI over 

M q ™/TH UOti0nal Media b3Sed UP ° n Aldernan at ^ (1978) and LgLson's 
stu rf ^4f U f eS * *t boated, the previously discussed assumpSionl abou? 
students' desires for CBI courseware have been based on these two studies a n d 

parallel^esultff 16 ^ ^ Jenklna & ^*>t, 1981) which obtained 

suggest that sSd^J nS w"?H lar Mthod <*<*i«- Nonetheless, these studies do 
suggest that students would accept taking a course by CBI. 

ed fSf^E!! 'J C1978) and lfa «""« , » (1978) findings have also suggest- 

ed that students do not believe that CBI is dehumanizing as an educational 

aboutlBrand^h 6 (1975 > P roVlded *°™ aubstantial evidence for this point 
ttt J possible dehumanization threat. After thoroughly reviewing 

J2f enr % °f; llan CBI "terature on student attitudes, Hng concluded 
that CBI is not a dehumanizing instructional experience. concluded 



B «.«.« f 1 - 8 (19 I 5 ? alS ° examined the data on the relationships between student 
attitudes and individual differences. She concluded that^earner control h 
not been shown to significantly influence student attitudes. So^v^soae 
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limited evidence wis found showing that computers motivated underaohievers a 
slower students, and potential dropouts* Mevareeh and Rich (1985) also dem- 
onstrated that Gil oould improve disadvac^taged students* perceptions of 
school life. Again, these studies did jig— >t systematically compare the rela- 
tive effects of CBI and conventional Insffliruction upon problem students 5 atti^ 
tudes. 

Psychometric Attributes of Attitudin al faaaurea 

Attitudinal data may be inconsistent because of problems with some of the 
instruments used in measuring these data** . j&ierr and Nawrooki (1978) have 
noted that most attitudinal instruments «ra designed ad hoc with neither the 
items not the metric properties of the aw-ales described. As stated by King 
(1975)i 

Most (attitudinal) studies are e3CPerMJaental-constructed tests 
which have unknown or unreported r e lrriabilities . (p7 ) 

For example, Jenkins and Dankert (i9&61) indicated that reliable data were 
collected; however, descriptions of the litems and the test-retest reliabili- 
ties were not provided. And Bangert^DrOs-Kns et al. (1985) and Kulik et al. 
( 1980) failed to describe the reliabiiit^d,es of the different attitudinal 
measures reviewed in their reports* 

Several of the previously cited studr-i es (e.g. , Avner, 1981 ; tollman et 
al. 19771 Knerr Oawrocki 1978; Long©, 1972| Loyd £ Gressard 1984b and 
Mevareeh & Rich* 1985) have developed o? used attitudinal measures with ap- 
parently sufficient reliabilities, M©\rai3*eeh and Rich measured students 1 per- 
ceptions of the instructional environment^ with a questionnaire developed by 
Epstein and Mcpartland (1976). This ques^stionnaire had a known internal con- 
sistency coefficient of .84. Long© repOizzrted nearly identical attitudinal 
scores on the saas items for two separstte^ studies $ which indicates that the 
measures were reliable* Perhaps* than, Q^stential reliability problems with 
attitudinal measures can be eliminated using established questionnaires or 

by replicating the results. Other ttmn * the few studies cited in this report 
(e.g*, Dallman etal's, 1977 and Longo * a ), a lack of replicated findings 
comparing CBI toother media exists in tChis research literature. 

Avner (1972) argued that reliability for attitudinal measures should not 
be determined by using a large number of " items selected for internal consis- 
tency. Rather, a highly reliable single item should be used for each attitude 
to be measured. He has claimed that r*ed£-uoing the number of items allows the 
researcher to sample student opinions rfor^rs frequently without encroaching on 
the students' goodwill or the limitaci o^-iass time available* And reliability 
measures which are based upon internal e^ensistenoy can be a misleading indi- 
cation of the questionnaire's value, b©Q«mse such a group of items can only 
be vaguely interpreted. Except for & fe^v studies conducted by Avner and 
associates, the reliability of single it-_em attitudinal measures has rarely 
been determined. 
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„„h r ° Inr 3 my ° ° XLSt With the val idity of the attitudinal measures 
used w CBI research, Discrepancies between students' responses to question- 

^nfl!^ 0bJe ° tlVe data haVS b " n foUnd by toe rr & Nawroeki ({978)1 
^ (1985)l W °u d and PitZ (1985) » Shleehter, for example noticed 

flat Snv ie r%H e ?° 6d that USlnS 3 light Pe " TOS ^latively eaey, when in 
fact many of their errors related to problems with using this responding 
mechanism. Additional efforts to compare self-reports with objective per- 
formance measures seem warranted. As stated by Avner (1972): 

While the questionnaire provides a convenient and useful method 
of gathering student data in certain situations (e.g., students' 
learning to use a system) there is no alternative to direct 
observation of student behavior (p5), 

f h 4- Be fff° ,e ^ (Personal communication, October 31, l 9 85) has suggested 
tnat attitudinal data must be validated by students' actual use of an imple- 
mented system. *^ 

Another threat to CBI attitudinal measures is that items regarding social 

J 7 aP L? 0t lncluded * This P^blem is especially important for 
studies (e.g.. tollman et al. 1977; and Tatsuoka and Misselt, 1978) in which 

» n l«Z l T7 ? aV K,?? en Part ° f the evaluation process. Reasons for including 
a social desirability measure will be elaborated upon in the next few pages! 

Problematic attitudinal measures are frequently found in the CBI litera- 
5" re : J S indicated, psychometric problems with these measures were observed 
in studies with favorable and in studies with neutral outcomes about 
student's preference for CBI. These measurement problems tend to further 
limit conclusions about CBI and student attitudes. 



Factors Affecting Student Attitudes 

in^^f llr \ ft "\ t0r affecti »K students' attitudes should be the CBI program's 
instructional value. Gieason (1 9 8l) has claimed that students do react very 
positively to good CBI programs and reject poor programs. Clement (1981) has 
also noted that CBI lesson materials which reduce student error rates may be 
beneficial to maintaining and producing positive attitudes. Clement and 
Gieason however, do not provide any data supporting their positions. 

Limited evidence exists about the effects of performance upon student 
attitudes. King (1975) reported only one study (Mitzel & Wodtke, 1965) which 

fou^d tL, 6 I 1 "if S \ ±P betW6en attltude s ^d performance. mt»i1 Wodtke 
found that a significant positive correlation existed between these varia- 
bles. On the other hand, Tatsuoka and Misselt (1978) could not find any 
substantial relationships between military students' gain-scores on a CBI 
training Program and attitudinal measures. This study investigated students' 
CBI use and their attitudes toward four different courses at Chanute Air 
Force Base. Correlations between students' gain-scores and attitudinal in- 
dexes of instructional effectiveness and positive affect were not significant 
at levels of .19 and .21, respectively. Significant correlations (p f .ol) 
01 .32 and -.29 for the instructional effectiveness and positive affect 
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variables, respectively, were found when a multiple regression technique was 
used to determine the correlational coefficients* The .19 and .32 correla- 
tions dealt with students 1 perceptions of PLATO'S effectiveness and perform- 
ance while the other two correlations involved students 1 positive feelings 
toward CBI and performance. Tatsuoka and Misselt never explained the reason 
for the directional shift in the second set of correlations* Also, as re- 
ported. Alderman et al. (1978) found that students had less favorable atti- 
tudes toward TICCIT than toward conventional instruction, but TICCIT was 
found to be more effective for some educational measures. A link then has 
not been found between attitudes and instructional quality. 

A link does seem to exist between users* attitudes and human factors 
issues (Avner, 1986, King, 1975j Magidson, 1978; McVey, Clauer , £ Taylor, 
1984; Shiechter, 1985; and Tatsuoka & Misselt, 1978). Avner found correla- 
tions of .80 between users' (28 instructors) ratings and system reliability. 
He also found a significant point-biseral correlation of .593 between in- 
structors 1 ratings and the decision to continue/discontinue the use of the 
PLATO system, Tatsuoka and Misselt found correlations of .-43 and -.^5 be- 
tween students 1 perceptions of mechanical failures and students 1 attitudes 
toward the system/ s acceptability and instructional effectiveness, respec- 
tively* McVey et al., discussed the possibility that hardware features might 
affect users' comfort. As stated by McVey et ai.i 

A thorough review paper by Campbell and Burden (1983) suggests 
that there are many overlooked factors contributing to the hu- 
morous accusation that VDT (CRT) means "Visual Discomfort Termi- 
nals. " (pi) 

Other reports from the human factors literature (e.g., Dainoff, Bapp, h 
Crane, 1981; IBM, 1984; Helander § Rupp, 1984 j and Rupp, 1981) have also 
discussed problems with CRTs. Dainoff et al. found that visual fatique was 
more pronounced among typists using CRTs than for typists using print-based 
terminals. They also noted that these differences in visual fatique were not 
related to amount of work or other job pressures* 

Different CRT standards have been recently developed by different coun- 
tries and agencies, e.g., Department of Defense (Pop) to control for possible 
problems with the design of these terminals (Helander & Rupp, 1981). Unfor- 
tunately, as noted by Helander and Rupp, the PoD standards have emphasized 
productivity over operator comfort and convenience. Possible instructional 
inefficiency, however, may be associated with "visual discomfort terminals" 
as students may not be able to use these terminals for an extended period of 
time. Reliable attitudinal data are needed for ascertaining specific problems 
that students may have with the CBI system. 

Useful information about CBI programs has been obtained when attitudinal 
measures have been used to ascertain specific issues. Tatsuoka and Misselt 
(1978) found that students 1 responses to open-ended attitudinal questions 
were helpful in revising poorly developed CBI lessons. Avner (personal com- 
munication, November 1985) discovered through such open-ended responses posi- 
tive attributes of CBI, such as scheduling flexibility for students, which 
would not have been discovered through other procedures. And Shiechter 
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(1985) found through students' interview responses that a particular svstem 

' At'fitud 6 I" USad " 8 to the conventional instructional program 

Atwtudmal measures, especially open-ended questions, may then be moat £™ 

^ , PrOV ^ ±nS f ormative feedback about students' perceptions If a ays! 
tern's strengths and weaknesses, y 

,„H/ UCt «K S ° ther than the CB1 s y at em may unduly influence students' atti- 
tudes. Abrami and Mizener (198?) noted that teacher attitudes had a substan 
Avne, e fioftl,T " ntUdentS ' attlt «de S in conventional instruction courses? 
Avner ( 1985) demonstrated that teachers' attitudes toward CBI did effect 
students' attitudes toward the system, Avner used exemplary and mediocre CB1 

dentfln^- f? JUd f d by * Panel of CBI Professionals, He Lund thaJ stu- 
dents initially judged both programs to be either positive or negative de- 
pending upon the teachers' positive or negative feelings toward the system" 
An examination of the data, however, indicated that students who had teachers 
w lt h positive attitudes toward CBI still-after a month's period-maintained 

ESSE more^^t" 0 ^^^ 116 mediOCre Pr ° Sram - A1So ' ^ese studenL haa 
slightly more positive attitudes toward CBI than the students in the 

^n P J ar T CB ? "f^iala-hostile teacher condition. Teachers' altitudes could 
then confound students' CBI attitudinal data, ituaes couia 

Because of the unique relationship between military instructors and their 
students, evaluations of military CBI programs would be especiall/prone to 
problems associated with instructor attitudes. Military instructors need to 

c7gST°ST f tha „ eva ^ atl0n PrOCeSS - Draxl and Aeeen Cl 9 8i>™SS Gray 
lilt inX de3Cr3b ^ the need to Sive briefings to enlist military instruc- 
tors interest m, and support of, new instructional systems. Military in 
struetors' attitudes-as civilian instructors' attitudea~may also 17 
S5ip e ?n!f y . superiors' beliefs. Indeed it is hard to imagine how 

these instructors can remain neutral toward CBI when their superiors el 
Sullivan (1985), have indicated that this medium will be an integral' parfof 
all future Army training. Such briefings by CBI e valuators and Army staff 
S a n K POS1 ? VS attitudes in instructors, while militating against 

f ""biased reports from students. Military students may be more 
Lilian students to state their dislike of a CBI system? 
because military instructors have much more power over their students than ri n 
conventional teachers. This possible difference between miLtary and civil! 
lan students can explain why more favorable comments toward CBI have blen 
found in the military training literature than in the civilian IlJeraSrS. 

Another possible confounding in attitudinal research is novelty effects. 

fni^ralinS^Tf 6 '^ ClaPk (1983 and 1985a) has cla ^d that students ar^ 
titnt IL I d / nd enthusi "tic to learn from a new CBI program. Ques! 
tions, though, remain about a system's ability to sustain students' interest 
and motivation. Pagliaro (1983), in a year-long study of a CBI program for 
S«nT??° 10 ?^ S ? Udants > found that these students reported the sysST To be 
significantly less interesting across testing periods. The magnitude of the 
decrements in students' ratings, however, was not great. 

Other CBI reports (e.g., Avner, 1985! Ford et al. 1972! King, l 975 . and 
Kagidson, 1978) have not been able to find any substantial effects for'cBI 
experience upon attitudinal data. Ford et al/found that long-term cZ users' 
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attitudes toward CBI were mora positive than the attitudes of first-time 
users. The long-time users were fifty students out of more than 200 sampled 
who had prior experience with CBI. Avner showed that subjects across the 
different experimental conditions reported more positive attitudes toward the 
CBI system with increased usage* Thus, novelty effects as defined by length 
of CBI experience do not seem to have a major influence upon students 1 atti- 
tudes. 

Clark and hir associates (e.g., Clark & Salomon, 1985) also suggested 
that students would prefer any newer instructional technologies over the 
established ones — "program novelty effects." Support for "program novelty 
effects" comes from Simonson's (1980) extensive review of the CBI literature. 
Simonson has concluded that students seemingly prefer to learn from any new 
instructional mode, e.g., instructional T.V. or programmed instruction, over 
conventional instruction. Shlechter (1985) reported that students preferred 
a CBI system to conventional instruction because of the individualized in- 
struction associated with this system. These students would then seemingly 
prefer any individual mode of instruction over conventional instruction. As 
has been previously argued, individualized instruction characterizes nearly 
all modern instructional innovations. 

Juola (1977) reported findings which conflicted with those previously 
cited about new media and student attitudes* Seven hundred freshmen at 
Michigan State University found instructional television and programmed in- 
struction to be undesirable instructional media. Juola's study, however, is 
not typical of this research literature. Future research is needed in which 
students 1 CBI attitudes are compared with attitudes for other i (a) equiva* 
lent instructional media and (b) new instructional modes. 



Conclusions 

Inconsistent and confounding findings are predominate in the CBI attitud- 
inal findings. Furthermore, these problematic findings seem to relate to 
factors other than courseware differences. CBI systems should not be imple- 
mented based primarily upon attitudinal data. Recommendations have also been 
made about using attitudinal data for formative feedback, making attitudinal 
comparisons with other new and equivalent instructional technologies, using 
objective measures to validate the subjective attitudinal measures, and con- 
ducting more naturalistic attitudinal studies. Such naturalistic studies 
would ultimately provide CBI professionals with the most useful data about 
CBI and student motivation. 
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Summary and Recommendations 



CBI o empirical support does not exist for the claimed advantages of 

CBI over other instructional media fori a) reducing training time; b) reduo- 

maferials^dfaef tS| H° - faCll i tatlnS Students ' »«tery of tie instructional 
^ er a J S V d accommodating individual learning differences; and e) metivat- 

„"o? £K?v f arni ^: ThS laek ° f support for these issues is 

ware lit J explained by problematic courseware. As discussed, some course- 
benefits ITclT / ° f the finCSt -"actional designers. And some 
Deneiits of CBI, e.g., training-reductions, have been attributable to in- 

medi? °?RT.f at r eS f ±Ch Jnher ' ent ln individualized instructional 
SS^STbSET SUpe — as a instructional medium has thus 

Tr B T N °! WithStandinS thSSe mixed Endings, future generations of CBI, e.g. 
ICBI, do promise to have a significantly positive impact upon students' 

nn?o^, lV % P r OGeSSe S; ^ m8dlUn also Promises to help slow learners. 
CM anTslow^ear ° blemat j? researeh Praetieaa prohibit any conclusions about 
inLt , ? , learnerB - Also > CBI does not seem to be a dehumanizing 
instructional experience. 

^f.™» Dle M a ^ e ^ search Procedures were also found throughout the CBI lit- 
lue trdifferL n0tl r able ,° f the " e res ^ h Probl«ns were; a) confoundings 
nroDrLte i instructional content; b) making comparisons with Sap- 

^ef^Pn J C) n C ° nfOUndlnSS dUS t0 B P r °Sram novelty effects" and 

teacher attitudes;" and d) findings which were not replicated. Recommenda 
tions have also been made that future CBI evaluations a) be conduced on 
fblly implemented systems; b) examine hidden life cycle costs for each imple- 
cos£ ^% e) eXanlne , the effee ts of human factors variables upon a system's 
cost-effectiveness; d) examine the effects of human factors variables upon 
students' comfort; e) examine students' abilities to transfer the instrSc- 
eJated SSTS '* * ■ "T"" the ""warranted additional instructions asso- 
confounrtinL 5 e *P e ^mental or control treatments; g) control for possible 
bldZn EEL Si t ° v , lnSUfficlent ^sting durations; h) examine interaction! 
between students' characteristics and specific delivery features and CBI 
systems; and i) use attitudinal data for formative but not sumoative evalua- 
Won. Clearer answers about CBI's inherent value-in general and for spe- 
cific system' s-would be obtained if future CBI investigations deal with the 
research problems and recommendations cited in this paragraph. 

atln f 8 T f P * r ' S Bethodologioal issues reflect the complexities in CBI evalu- 
ation. Ideally, CBI evaluations should be done by a team of professionals 
from various disciplines-military or civilian education, human factors 
infand exLr yeh0l ° ey ' in8t ™tional technology. Also some time consu„= 

CbI Valuation Is ^^T^ proeedures raust be employed. To the extent that 
CBI evaluation is used to compare CBI to other instructional media, especiai- 

shoSdle collLtaf i0n ' ^ b ° th ^"-"etionai-ind-ionglSaiSnS" 
should be collected. A cross-sectional design is needed for initial assess- 
ments of students. CBI performance, while longitudinal data are necessary tl 
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ascertain the long-term learning and attitudes associated with the CBI sys- 
tem* The longitudinal data can also be used as a reliability check. System- 
atic programs are needed in which priorities are assigned to independent 
variables and variables are systematically manipulated and measured in suc- 
cessive studies using the same CBI system. The systematic analyses would 
provide information about the relationship between estimated and life cycle 
costs* Also programmatic research is needed to provide further insights into 
the question of whether the cited CBI limitations are due to courseware or 
hardware problems. 

Another issue in CBI evaluations concerns the traditional use of forma- 
tive evaluations for debugging the CBI*s programs problem areas* Williges, 
Williges, and Elkerton (in press) have noted that formative evaluation is 
central to the design of any computer system. They fail, however, to mention 
the necessity for conducting a similar formative evaluation upon the existing 
or comparative instructional system(s). These discrepant evaluation proce- 
dures will certainly lead to the possibility of making confounded comparisons 
because the CBI program is operating at optimal efficiency while the same 
cannot be said for the comparative media. 

Formative evaluations should also be conducted after the initial experi- 
mental data have been collected, rather than as traditionally done prior to 
the experimental process. Such formative evaluation data could help deter- 
mine the reasons for the experimental findings. Furthermore, the formative 
data could also help determine the research programme next phase* As sug- 
gested by Boldovioi and Scott (198*0, ^valuators should thus initially deter- 
mine the CBI system's relative efficiency and then pinpoint the reasons that 
the system was more, less, or equally effective than was the comparative 
media. The evaluator f s role is to assess objectively a system's instruction- 
al value and not to help with the design process. 

Financial j> temporal, and personnel limitations may prohibit the extensive 
evaluation procedures recommended in this report. Needless CBI studies could 
be eliminated if the evaluation team could build upon the works done by other 
teams on that system. Presently, communications problems among CBI profes- 
sionals present an obstacle to this building process* Many previous evalua- 
tions of CBI systems are not easily available. Bangert-Browns et al f s (1985) 
meta-analysis of the CBI literature included only six published studies out 
of more than forty studies examined. This author also found that many CBI 
reports could only be found through corporation and military archives* And 
some studies which showed unfavorable results regarding a CBI system's effec- 
tiveness were not available for distribution. 

A CBI handbook, such as Euros 1 tental Measurement Yearbooks, is needed to 
provide a common information source for CBI professionals about different CBI 
systems and corresponding evaluations* Ibis handbook should include a de- 
tailed description of the studied sample, research instruments, research 
procedures, and findings* Such information would help prospective CBI 
implementers to determine the best system for their instructional needs, or 
to discover that previous CBI evaluations might not be transferable to their 
situations. The military has tried to establish such a CBI library with the 
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TRIADS program (see Ballman et al., 1983 for a description of this program)- 
evaluSiofdata. e0nSlPtS ° f & °' ^ ™™* ~ ao£S£££' 



«v«.*2 J f 5° r ? main SbOUt the °P tlnal situations for implementing a CBI 

fnstruotioLf^ff thr °^ hOUt reP ° rt ' CB1 may be bes * used af an " 

th!i 1 ° - SUpplemcnt the established instructional program rather 

JSSp"^ Prl r Vy in3trUCtionaJ ^t, using CBI to supplement 

rather than replace other instructional modes raises further questions about 
the costs associated with this medium. If CBI can help problem learners than 
it would be worth the additional expenditures. Uian 

_ m !?f sltuati °nal aspect involves determining the appropriate CBI sys- 

si l L/ rU f l0nil situation. The Army, for example, must decide on 
such options as using a hand-held tutor or microcomputer system to teach 

fh- ?h ?.; hlf L fOQU ! fron Wie-tioM of the inherent superiority of CBI to 
the identification of conditions under which CBI and alternative media pro- 
duce and do not produce the desired results. Various media have various 
strengths which must be first enumerated and then matched with intended in- 
structional settings, objectives, and resources. ^ aea in 

Widespread implementation of CBI for all educational situations is not 

Sls'Vn^ruot-nn 0 ? t Tld i RCi : imlnatC USe ° f CB1 °° Uld lead to the abandonment of 
this instructional technology, because the research evidence has indicated 

hi .,«ft ? rea Premises may outweigh this medium's potential. CBI may 

M*™? BS ^ anstrUCtional t001 whioh can h elP make established instruc- 

sttuar^n^rf T°v, feCtlVe - And CB1 ^^hers must help determine those 
situations for which instructors can best use this medium. 



Conclusions 



In closing, CBI's effectiveness over other alternative media for military 
training has yet to be established. CBI can provide the military with 
standardization of instruction and provide reserve components with needed 
sustainment training. Also future generations of CBI, especially ICBl-s, 
with Potential advances in hardware, software, and courseware design could 
make this medium more effective. The military research community must be 
more careful in conducting future CBI evaluations and must be careful not to 
overstate the promises of this medium. Researchers of military training 
?or S CM S J f ° CUS UPO " dete ™ lnin S the learning tasks most suitable 

fn JS' S ?° U ^ ? oncentrate on helping the military education community 

to most effectively integrate computers in. already established instructional 
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Final Implications 



1 * Consistent evidence does not exist to support the instructional superior- 
ity of CBI over other media. 

2* Future generations of CBI, such as IOBI, promise to help students* cogni- 
tive processes. This medium also promises to help slow learners. Unfortu- 
nately, problematic research practices prohibit any conclusions about CBI and 
slow learners* 

3. Problematic research procedures are found throughout this literature. 

4. The methodological recommendations discussed in page 31 must be incorpo- 
rated in future CBI evaluation. 

5. The evaluation team ideally would consist of professionals from various 
disciplines • 

6. Programmatic research on CBI systems should be conducted which includes 
both cross-sectional and longitudinal studies. 

7* Formative evaluation should be conducted on the different media being 
compared • 

8. Formative evaluation should be conducted upon completion of the initial 
experimental study. 

9. A CBI handbook should be developed which contains information about pre- 
vious CBI evaluations. 

10. CBI researchers should shift focus from examining the inherent superior- 
ity of this medium to identifying conditions for using computers in the in- 
structional process. 

11* Widespread implementation of CBI for all educational situations is not 
recommended * 
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